Autonomous Unmanned Aerial Vehicles Research Articles

Visual SLAM for Unmanned Aerial Vehicles: Localization and Perception

Localization and perception play an important role as the basis of autonomous Unmanned Aerial Vehicle (UAV) applications, providing the internal state of movements and the external understanding of environments. Simultaneous Localization And Mapping (SLAM), one of the critical techniques for localization and perception, is facing technical upgrading, due to the development of embedded hardware, multi-sensor technology, and artificial intelligence. This survey aims at the development of visual SLAM and the basis of UAV applications. The solutions to critical problems for visual SLAM are shown by reviewing state-of-the-art and newly presented algorithms, providing the research progression and direction in three essential aspects: real-time performance, texture-less environments, and dynamic environments. Visual–inertial fusion and learning-based enhancement are discussed for UAV localization and perception to illustrate their role in UAV applications. Subsequently, the trend of UAV localization and perception is shown. The algorithm components, camera configuration, and data processing methods are also introduced to give comprehensive preliminaries. In this paper, we provide coverage of visual SLAM and its related technologies over the past decade, with a specific focus on their applications in autonomous UAV applications. We summarize the current research, reveal potential problems, and outline future trends from academic and engineering perspectives.

A novel method for measuring roll angle

The precise measurement of the spin speed of a high-speed autonomous unmanned aerial vehicle (HSA-UAV) is a key element in mastering flight stability, and the measurement of roll angle is the key to determining the accuracy of navigation control systems. We put forward a novel method for measuring roll angle. This method starts from the time–frequency domain analysis of the output signal of the geomagnetic sensor, extracts the time–frequency ridge of the time–frequency matrix (TFM) to obtain the spin speed of the HSA-UAV, and reconstructs the output signal of the geomagnetic sensor. It can calculate the roll angle when the calibration parameters of the geomagnetic sensor are unknown and have good engineering practical value. In addition, we also propose an improved nonlinear short-time Fourier transform with high-frequency resolution and a forward penalty dynamic path ridge-extraction method with frequency jump suppression to extract instantaneous frequency in the TFM.

A fast, lightweight deep learning vision pipeline for autonomous UAV landing support with added robustness

Despite massive development in aerial robotics, precise and autonomous landing in various conditions is still challenging. This process is affected by many factors, such as terrain shape, weather conditions, and the presence of obstacles. This paper describes a deep learning-accelerated image processing pipeline for accurate detection and relative pose estimation of the UAV with respect to the landing pad. Moreover, the system provides increased safety and robustness by implementing human presence detection and error estimation for both landing target detection and pose computation. Human presence and landing pad location are performed by estimating the presence probability via segmentation. This is followed by the landing pad keypoints’ location regression algorithm, which, in addition to coordinates, provides the uncertainty of presence for each defined landing pad landmark. To perform the aforementioned tasks, a set of lightweight neural network models was selected and evaluated. The resulting measurements of the system’s performance and accuracy are presented for each component individually and for the whole processing pipeline. The measurements are performed using onboard embedded UAV hardware and confirm that the method can provide accurate, low-latency feedback information for safe landing support.

Dynamic Topology Organization and Maintenance Algorithms for Autonomous UAV Swarms

Dynamic Topology Organization and Maintenance Algorithms for Autonomous UAV Swarms

VizNav: A Modular Off-Policy Deep Reinforcement Learning Framework for Vision-Based Autonomous UAV Navigation in 3D Dynamic Environments

Unmanned aerial vehicles (UAVs) provide benefits through eco-friendliness, cost-effectiveness, and reduction of human risk. Deep reinforcement learning (DRL) is widely used for autonomous UAV navigation; however, current techniques often oversimplify the environment or impose movement restrictions. Additionally, most vision-based systems lack precise depth perception, while range finders provide a limited environmental overview, and LiDAR is energy-intensive. To address these challenges, this paper proposes VizNav, a modular DRL-based framework for autonomous UAV navigation in dynamic 3D environments without imposing conventional mobility constraints. VizNav incorporates the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm with Prioritized Experience Replay and Importance Sampling (PER) to improve performance in continuous action spaces and mitigate overestimations. Additionally, VizNav employs depth map images (DMIs) to enhance visual navigation by accurately estimating objects’ depth information, thereby improving obstacle avoidance. Empirical results show that VizNav, by leveraging TD3, improves navigation, and the inclusion of PER and DMI further boosts performance. Furthermore, the deployment of VizNav across various experimental settings confirms its flexibility and adaptability. The framework’s architecture separates the agent’s learning from the training process, facilitating integration with various DRL algorithms, simulation environments, and reward functions. This modularity creates a potential to influence RL simulation in various autonomous navigation systems, including robotics control and autonomous vehicles.

On the design of deep learning-based control algorithms for visually guided UAVs engaged in power tower inspection tasks.

This paper focuses on the design of Convolution Neural Networks to visually guide an autonomous Unmanned Aerial Vehicle required to inspect power towers. The network is required to precisely segment images taken by a camera mounted on a UAV in order to allow a motion module to generate collision-free and inspection-relevant manoeuvres of the UAV along different types of towers. The images segmentation process is particularly challenging not only because of the different structures of the towers but also because of the enormous variability of the background, which can vary from the uniform blue of the sky to the multi-colour complexity of a rural, forest, or urban area. To be able to train networks that are robust enough to deal with the task variability, without incurring into a labour-intensive and costly annotation process of physical-world images, we have carried out a comparative study in which we evaluate the performances of networks trained either with synthetic images (i.e., the synthetic dataset), physical-world images (i.e., the physical-world dataset), or a combination of these two types of images (i.e., the hybrid dataset). The network used is an attention-based U-NET. The synthetic images are created using photogrammetry, to accurately model power towers, and simulated environments modelling a UAV during inspection of different power towers in different settings. Our findings reveal that the network trained on the hybrid dataset outperforms the networks trained with the synthetic and the physical-world image datasets. Most notably, the networks trained with the hybrid dataset demonstrates a superior performance on multiples evaluation metrics related to the image-segmentation task. This suggests that, the combination of synthetic and physical-world images represents the best trade-off to minimise the costs related to capturing and annotating physical-world images, and to maximise the task performances. Moreover, the results of our study demonstrate the potential of photogrammetry in creating effective training datasets to design networks to automate the precise movement of visually-guided UAVs.

UAV control in autonomous object-goal navigation: a systematic literature review

Research interest in autonomous control of unmanned aerial vehicles (UAVs) has increased rapidly over the past decade. They are now widely used in civilian, military, and private areas. Applications include surveillance, search and rescue, and delivery tasks. More broadly, they excel at solving problems where a significant amount of space must be covered and traveled. However, using UAVs to solve navigation problems with full autonomy necessitates the mastering of complex subtasks. A solution that includes control, planning, localization, and mapping remains an open challenge. Object-goal navigation contains the same navigation problems where the main objective is to reach a target object. The search and identification of this target are central to the vehicle’s navigation. This requires an understanding of what it is and where it can be located to move around the scene. This paper presents a systematic literature review on object-goal navigation and its subtasks, using autonomous UAVs. Survey taxonomies were found for the tasks and methods behind navigation and target localization problems using UAVs. The review analyzed 67 articles found between 2011 and 2022. They were found in the ACM, IEEE Xplore, WebOfScience, Science Direct, and Scopus databases. This review revealed essential issues related to autonomous navigation task dependencies. Moreover, it highlighted gaps in UAV development and framework standardization. Open challenges for autonomous UAV control for object-goal navigation must address the research on finding methods for problems. For example, autonomy level and comparison metrics, considering safety, ethics, and legal implications.

Autonomous UAV Networks Based on Artificial Intelligence

Autonomous UAV Networks Based on Artificial Intelligence

Vision-state Fusion: Improving Deep Neural Networks for Autonomous Robotics

AbstractVision-based deep learning perception fulfills a paramount role in robotics, facilitating solutions to many challenging scenarios, such as acrobatic maneuvers of autonomous unmanned aerial vehicles (UAVs) and robot-assisted high-precision surgery. Control-oriented end-to-end perception approaches, which directly output control variables for the robot, commonly take advantage of the robot’s state estimation as an auxiliary input. When intermediate outputs are estimated and fed to a lower-level controller, i.e., mediated approaches, the robot’s state is commonly used as an input only for egocentric tasks, which estimate physical properties of the robot itself. In this work, we propose to apply a similar approach for the first time – to the best of our knowledge – to non-egocentric mediated tasks, where the estimated outputs refer to an external subject. We prove how our general methodology improves the regression performance of deep convolutional neural networks (CNNs) on a broad class of non-egocentric 3D pose estimation problems, with minimal computational cost. By analyzing three highly-different use cases, spanning from grasping with a robotic arm to following a human subject with a pocket-sized UAV, our results consistently improve the R$$^{2}$$ 2 regression metric, up to +0.51, compared to their stateless baselines. Finally, we validate the in-field performance of a closed-loop autonomous cm-scale UAV on the human pose estimation task. Our results show a significant reduction, i.e., 24% on average, on the mean absolute error of our stateful CNN, compared to a State-of-the-Art stateless counterpart.

Autonomous Navigation UAVs for Enhancing Information Freshness for Reconnaissance

The demand for autonomous navigation UAVs in reconnaissance is steadily increasing. One crucial aspect of these missions is the freshness of reconnaissance information, which serves as a vital indicator of mission effectiveness. However, there is a lack of targeted investigation in the research on autonomous single/multi-UAV missions and joint path planning. Furthermore, the use of visual–inertial odometry (VIO) in rotary-wing UAVs can lead to significant positional drift during flight, which may result in the loss of the UAV and mission failure. This paper investigates joint planning problems in single/multi-UAV reconnaissance missions under both GPS-available and GPS-unavailable scenarios and proposes an integrated data collection and beacon-assisted localization approach. Finally, the numerical results demonstrate the effectiveness of the proposed scheme in enhancing the freshness of reconnaissance information.

Skybound Intelligence: AI's Impact on Drone Technology

The integration of Artificial Intelligence (AI) with drone technology has revolutionized various industries, from agriculture and construction to defense and emergency response. This paper aims to provide an overview of the multifaceted role AI plays in enhancing drone capabilities and addressing challenges in different domains.Firstly, the paper discusses how AI-driven autonomy enables drones to perform complex tasks autonomously, such as navigation, obstacle avoidance, and mission planning. Advanced algorithms, including machine learning and computer vision, empower drones to interpret and respond to dynamic environments in real-time, significantly expanding their operational capabilities.Secondly, the paper examines the crucial role of AI in data processing and analysis for drones. AI algorithms enable drones to collect, process, and analyze vast amounts of data quickly and efficiently, extracting actionable insights for various applications. In agriculture, for instance, drones equipped with AI can analyze crop health, identify pest infestations, and optimize resource allocation for improved yields.Furthermore, the paper explores how AI enhances the security and safety of drone operations. AI-powered systems enable drones to detect and mitigate potential risks, such as unauthorized access, airspace violations, and mechanical failures, thereby ensuring safer and more reliable operations, especially in densely populated areas or sensitive environments. Moreover, the paper discusses the implications of AI in enabling swarming capabilities among drones. Through collaborative AI algorithms, drones can coordinate their actions, communicate intelligently, and achieve collective goals efficiently, leading to applications like search and rescue, surveillance, and infrastructure inspection on a large scale. Lastly, the paper addresses the ethical and regulatory considerations associated with the integration of AI in drone technology, including privacy concerns, algorithmic biases, and the need for transparent decision-making processes. Keywords: AI-enabled drones, Autonomous UAVs, Drone autonomy, AI-driven navigation, Machine learning for drones, and Drone surveillance

A State-Decomposition DDPG Algorithm for UAV Autonomous Navigation in 3-D Complex Environments

A State-Decomposition DDPG Algorithm for UAV Autonomous Navigation in 3-D Complex Environments

Model-based impending lithium battery terminal voltage collapse detection via data-driven and machine learning approaches

Lithium battery is an important power source of an electrical vehicle (EV). Practically, when a battery is about to fail, it needs to be removed from the load because keeping it connected can lead to permanent damage. Hence, it is important to detect battery failure to sustain the lifespan of the battery and avoid safety issues which will contribute to the sustainability of the EV. However, determining the effective disconnecting or discharging moment of the battery remains a challenging issue in EV applications. In order to solve the abovementioned problem, a control framework and maximum likelihood estimation are proposed to estimate the parameters of a battery model. In addition, both the sparse representation and dynamic mode decomposition (DMD) approaches are applied to protect the physical battery unit from a failure scenario. For comparison purposes, the proposed framework is compared to a Lyapunov-based detection approach along with neural network (NN) and linear discriminant analysis (LDA). Moreover, several state estimation algorithms are applied to estimate battery state of charge (SOC) at each time step: extended Kalman filter, extended Kalman smooth variable structure filter, and cubature Kalman filter. Finally, the experimental results showed that the proposed DMD approach outperforms the sparse representation, NN, LDA, and Lyapunov-based approaches in terms of detection accuracy. Moreover, the proposed DMD strategy is more robust towards the inaccuracy of the estimated SOC. Besides that, the proposed battery model parameter estimation approach exhibited fast processing time; therefore, it is a reliable choice for recomputing the model parameters from time to time to compensate aging effect. An autonomous unmanned aerial vehicle (UAV) was simulated as a proof-of-concept in a MATLAB environment to verify the detection performance of the proposed methods. Both actual and UAV results showed that the DMD demonstrated more robust detection performance than other methods in terms of processing time and detection accuracy.

Forest in situ observations through a fully automated under-canopy unmanned aerial vehicle

ABSTRACT       Close-range sensing has yet to attain the status of being a dependable source for in situ forest information as the conventional field inventory. Each solution has its advantages and disadvantages in terms of accuracy, completeness, and efficiency. For a forest area, Terrestrial Laser Scanning (TLS) has the highest data quality, but is limited to static perspectives and lack the efficiency. Mobile Mapping Systems (MMS) systems gain on the efficiency but compromise the data quality. More recently, under-canopy UAV caught attentions for its potential to leverage the advantages of both TLS and MMS systems. This study demonstrates the feasibility of autonomous forest in situ investigation using an autonomous under-canopy UAV Laser Scanning (ULS) system, and evaluates the performance of such system in deriving key forest and tree attributes through a comparison with other close-range sensing systems such as the TLS and the Personal Laser Scanning (PLS). The under-canopy ULS system uses an onboard LiDAR sensor to aid its self-traverse in an unknown forest environment and to collect point cloud data during its movement inside the forest. Key factors influencing the systems’ overall performance were investigated via various experiments. The point cloud data collected by the under canopy autonomous ULS system deliver similar stem capturing capacity as TLS in single layer forest stands with less undergrowth. The RMSEs of the DBH estimates were 0.81 cm (3.80%), 4.12cm (19.92%), and 5.13cm (22.01%), respectively. The RMSEs of the stem curve estimates were 1.27 cm (5.48%), 3.97 cm (17.63%), and 5.18 cm (22.49%), respectively. The geometric accuracy and the completeness of the point cloud significantly improved when the trajectory was densified. More studies on autonomous route planning in complex unknown forest is required to improve the system mobility, data quality, and the applicability of such systems in future practical forest in situ observations.

SUITABLE LANDING SITE SELECTION FOR UNMANNED AERIAL VEHICLES USING AIRBORNE LASER SCANNING POINT CLOUD

Abstract. In the context of autonomous landing for unmanned aerial vehicles (UAVs), selecting a suitable landing site is crucial. This research presents a new method for automatically identifying safe landing sites based on point cloud data acquired through an airborne laser scanning (ALS) system. The proposed approach begins by detecting flat regions using principal component analysis (PCA) and region-growing algorithms. Subsequently, a terrain complexity assessment is conducted through plane fitting using an enhanced progressive sample consensus (PROSAC) algorithm. This assessment assists in identifying the most suitable landing site within a specified landing zone. The method's effectiveness is demonstrated through experiments conducted on two distinct natural terrains from the DALE dataset. The results show that the proposed approach can accurately classify landing zones and identify preferred sites that meet safety criteria. This study's findings underscore the proposed method's effectiveness and feasibility in improving the safety and reliability of autonomous UAV landings.

A Comprehensive Survey of Unmanned Aerial Vehicles Detection and Classification Using Machine Learning Approach: Challenges, Solutions, and Future Directions

Autonomous unmanned aerial vehicles (UAVs) have several advantages in various fields, including disaster relief, aerial photography and videography, mapping and surveying, farming, as well as defense and public usage. However, there is a growing probability that UAVs could be misused to breach vital locations such as airports and power plants without authorization, endangering public safety. Because of this, it is critical to accurately and swiftly identify different types of UAVs to prevent their misuse and prevent security issues arising from unauthorized access. In recent years, machine learning (ML) algorithms have shown promise in automatically addressing the aforementioned concerns and providing accurate detection and classification of UAVs across a broad range. This technology is considered highly promising for UAV systems. In this survey, we describe the recent use of various UAV detection and classification technologies based on ML and deep learning (DL) algorithms. Four types of UAV detection and classification technologies based on ML are considered in this survey: radio frequency-based UAV detection, visual data (images/video)-based UAV detection, acoustic/sound-based UAV detection, and radar-based UAV detection. Additionally, this survey report explores hybrid sensor- and reinforcement learning-based UAV detection and classification using ML. Furthermore, we consider method challenges, solutions, and possible future research directions for ML-based UAV detection. Moreover, the dataset information of UAV detection and classification technologies is extensively explored. This investigation holds potential as a study for current UAV detection and classification research, particularly for ML- and DL-based UAV detection approaches.

EFP: Efficient Frontier-Based Autonomous UAV Exploration Strategy for Unknown Environments

EFP: Efficient Frontier-Based Autonomous UAV Exploration Strategy for Unknown Environments

CHALLENGES AND BARRIERS FOR UNMANNED AERIAL VEHICLE (UAV) IMPLEMENTATION IN MALAYSIAN INFRASTRUCTURE PROJECTS

The usage of Unmanned Aerial Vehicles (UAV) in infrastructure projects has inclined drastic recently due to its benefits. However, there is a slowness in transition from conventional construction approach to new innovation. Besides, there are lots of issues in current practices such as cost overrun, dangerous, delay of time completion and lack of an overall comprehensive scientific system. Furthermore, although there was high level of awareness on the advantages of using UAVs, the Malaysian construction industry are still reluctant for the changes. In response, to overcome issues that the construction industry faces, such as infrastructure projects thus, this study aims to develop a conceptual framework for promoting the applications of UAV in Malaysia infrastructure projects. The methods adopted include explanatory sequential research methods. For instance, questionnaire surveys and interviews are used to collect quantitative and qualitative data, respectively. Besides that, a refined conceptual framework of the challenges and barriers of UAV adoption in Malaysia infrastructure projects were mainly contributed by the restrictive regulatory environment and industry point of view. For example, the privacy issues and guidelines on autonomous UAV and the industry lack of confidence in the adoption of UAV and education on the adoption of the UAV.  In addition, it offers some solutions for the issues that have been revealed in the preliminary conceptual framework for the challenges on UAV adoption in Malaysia infrastructure projects. For instance, amaze the contractor with the final data from the UAV.  Lastly, the identify solution served as a reference for other Malaysian construction stakeholders that have not used UAVs before. The recommendations for future scholarly research include the roles of UAV technology with other technologies such as Artificial Intelligent (AI) or integration of data with other software.

An oil and gas pipeline inspection UAV based on improved YOLOv7

This study proposes a method of autonomous navigation UAV for oil and gas pipeline (OGP) dial detection based on the improved YOLOv7 model. The canny edge detection algorithm is applied in identifying the edges of the pipeline, and the Hough transform algorithm is used to detect the pipeline in a straight line. The intelligent UAV P600 is guided to patrol the oil and gas dials (OGD) along the pipeline, and the trained improved YOLOv7-based model is adopted to identify the OGD data. Dial recognition is divided into two stages, that is, dial contour detection and dial reading recognition. For the dial recognition rate (RR), the Levenstein distance, a commonly used method, is introduced, thereby calculating the distance between two character sequences. Meanwhile, an integrated global attention mechanism (GAM) is proposed based on the YOLOv7 model, aiming at extracting more informative features. With this mechanism, the channel and spatial aspects of the features are effectively captured, and the importance of cross-dimensional interactions is increased. By introducing GAM attention mechanism in the backbone and head of YOLOv7, the network’s ability in efficiently extracting depth and primary features is enhanced. ACmix (a hybrid model combining the advantages of self-attentiveness and convolution) is also included, with ACmix module improved. The improved ACmix module has the objectives of enhancing feature extraction capability of backbone network and accelerating network convergence. By substituting 3 × 3 convolutional block with 3 × 3 ACmixBlock and adding a jump connection and a 1 × 1 convolutional structure between the ACmixBlock modules, E-ELAN module in YOLOv7 network is also improved, thus optimizing E-ELAN network, enriching features extracted by E-ELAN network, and reducing inference time of YOLOv7 model. As indicated by comparing the experimental results of the six model algorithms (improved YOLOv7, YOLOv7, YOLOX, YOLOv5, YOLOv6 and Faster R-CNN), the improved YOLOv7 model has higher mAP, faster RR, faster network convergence, and higher IOU. In addition, a generic real dataset, called custom dial reading dataset, is presented. With well-defined evaluation protocol, this dataset allows for a fair comparison of various methods in future work.

Trajectory Interception Classification for Prediction of Collision Scope between Moving Objects

In the fields of autonomous navigation and vehicle safety, accurately predicting potential collision field points between moving objects is a significant challenge. A novel computing technique to enhance trajectory interception analysis is presented in this paper. Our objective is to develop a field model that can accurately forecast collision zones, improving road transportation safety and the use of autonomous cars. Our main contribution is a binary classification model called PCSMO (Prediction of Collision Scope between Moving Objects), which is based on zero-shot learning. Gann angles, which are typically 45 degrees, are used to analyze the trajectories of moving objects. This method is inspired by GANN (Gann Angle Numeric Nomenclature). Compared to earlier techniques, this model more accurately identifies potential collision collision interception zones. The technique computes Gann angles for trajectory analysis and extracts GPS coordinates of moving objects from video data using OpenCV. It offers a more sophisticated comprehension of object movement patterns and points of interception. To assess the precision, recall, F1-score, and prediction accuracy of our model, we employ 10-fold cross-validation. Comparing the PCSMO model to existing models, these metrics demonstrate how well the PCSMO model predicts potential collision zones. Our approach, we discovered, enhances trajectory analysis—a critical component of safer autonomous navigation systems. With potential applications in autonomous vehicle and UAV safety, the PCSMO model improves field interception classification.